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Abstract. The Dynamic Occupational Therapy 
Cognitive Assessment for Children (DOTCA-Ch) is a 
tool for finding our about cognitive problems in school- 
aged children. However, the DOTCA-Ch was developed 
in English for Western children. For this reason, it’s not 
appropriate for Thai children because of the differences 
of culture and language. The objectives of this study 
were aimed at translating the DOTCA-Ch in Orienta- 
tion, Spatial Perception, and Thinking Operations sub 
tests to a Thai version on a World Health Organization 
back-translation process, and to examine its internal con- 
sistency, inter-rater reliability and test-retest reliability. 
Participants consisted of 38 intellectually impaired and 
learning disabled individuals between the ages of 6-12 
years. Results from this study revealed high internal 
consistency in the Orientation sub test (a=.83) Spatial 
Perception sub test (a=.82) and Thinking Operations 
sub test (a=.82); high inter-rater reliability in the Ori- 
entation sub test (ICC =.83), Spatial Perception sub test 
(ICC =.84) and Thinking Operations sub test (ICC =.74); 
and high test-retest reliability in the Orientation sub test 
(ICC =.84), Spatial Perception sub test (ICC =.86), and 
Thinking Operations sub test (ICC =.85). These results 
indicate that the Thai version of the Orientation, Spatial 
Perception, and Thinking Operations sub test might be 
used as an appropriate assessment tool for Thai children, 
based on psychometric evidence including internal con- 
sistency, inter-rater reliability and test-retest reliability. 
However, additional study of other psychometric proper- 
ties, including, predictive validity, concurrent reliability, 
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and inter-rater reliability during the mediation process of 
this assessment tool needs to be carried out. 

Keywords: Cognitive, Orientation, Spatial Per- 
ception, Thinking Operations, DOTCA-Ch. 


1. INTRODUCTION 


Cognition is the high level ability of 
the brain that can help children to learn to be 
adapted to any situation. This ability enables 
children to have complex motor planning 
skills, problem-solving skills, recognition 
skills, decision-making skills and memory 
process (Wittayakorn, 2004 and Shettleworth, 
2010). Cognition is especially important for 
younger school-age children aged 6-12 years 
who are in the “concrete operations” stage 
of Piaget’s cognitive development theory so 
that they are able to use logical and coherent 
actions in thinking and solving problems to 
understand and explain the concepts of things. 
For this reason, their cognitive function can 
make possible wider social interactions and 
development of more academic skills needed 
to learn to read, write, calculate, and manipu- 
late their hands in daily life activities (Chin- 
chai, 2000 and Encyclopedia of Children’s 
Health, 2014). Indeed, cognitive function is an 
important element in their daily lives, both in 
school and at home (Josman, 2005). However, 
children who have cognitive dysfunction will 
have problems with self-control, social par- 
ticipation, study, and participation in activities 
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in their school and community (Katz, 2005). 
Teachers and school therapists should know 
about their students’ cognitive dysfunction, so 
that students with cognitive dysfunction can 
be rapidly determined and provided with the 
appropriate intervention (Katz, 2007). 

All students who study in Thai govern- 
ment schools use Thai as a first language and 
English as a second language. However, there 
is the absence of standardized assessment 
tools in Thai to evaluate cognitive functions 
in school-age students. When therapists con- 
sidered that their pediatric clients need to be 
evaluated for cognitive function, the therapists 
will use non-standardized tests to evaluate 
the client’s performance. Besides using non- 
standardized tests, they sometimes adapt other 
standardized tests that were not developed to 
evaluate the cognitive function of children 
directly, for example: the Developmental Test 
of Visual Perception - 2nd edition for visual 
perception evaluation, and the Beery-Buk- 
tenica Developmental Test of Visual-Motor 
Integration — 6th edition for visual motor 
integration evaluation. When we explored the 
standardized assessment of cognitive func- 
tion, we found that many researchers utilized 
the Dynamic Occupational Therapy Cogni- 
tive Assessment for Children (DOTCA-Ch) 
(Rodger et al, 2005). 

The DOTCA-Ch, a criterion-referenced 
assessment tool, was developed to directly 
evaluate cognitive function in children ages 
6 — 12 years. It is implemented as a dynamic 
assessment to enable the identification of chil- 
dren’s learning potential and their thinking 
strategies (Katz et. al., 2004). Because the 
DOTCA-Ch serves as a foundation for the 
cognitive evaluation of children in an efficient 
and friendly format, and has the unique feature 
of the dynamic evaluation testing procedure, 
therapists use this assessment to find out about 
the cognitive problems of children and use the 
results to confidently plan intervention strate- 
gies tailor-made for the individual child under 
treatment (Katz, 2004 and Nowack, 2005).But 
the DOTCA-Ch was developed in western 
countries and that means that the therapists in 
Thailand are not familiar with it. In addition, 
some therapists try to use the DOTCA-Ch on 
Thai children but find it ineffective due to cul- 
tural and linguistic differences. 

In this study, we were concerned with 
developing a Thai version of the DOTCA-Ch. 
It is under a process of back-translation in the 
manual, scoring sheet, format and instruction 
and interpretation of data that are in English 
and studies about the psychometric properties, 


including internal consistency, inter-rater reli- 
ability and test-retest reliability. We chose to 
study three sub tests including orientation, 
spatial perception and thinking operations. 
These three sub tests were chosen because ori- 
entation is an important skill of basic cogni- 
tion that can be developed to meta-cognition, 
especially orientation of time. Munkhetwit 
(2008) reported that many studies showed 
trends indicating that orientation of time was 
a common problem for children. Because ori- 
entation of time relates to the memory of time 
and is a dynamic process, which continually 
happens, there is a need for an input process, a 
storage process, and a recall process. In addi- 
tion, problems of memory of time can be an 
indicator of memory impairment that has an 
influence on human occupations. The second 
sub test, spatial perception, is an important 
skill in activities of daily life. If children have 
a problem with spatial perception, they will 
have problems with using tools and with coor- 
dinating their bodily movements. The third 
sub test, thinking operations, is a basic skill 
for information processing that can help chil- 
dren to learn and understand the meaning of 
information, and to decide how to respond to 
that information (Munkhetwit, 2008). All of 
these sub tests are necessary to assess a child’s 
development of learning strategies and cog- 
nitive skills. The results of this research will 
be useful for therapists in Thailand in that it 
provides them with a standardized assessment 
methodology to accurately evaluate the cogni- 
tive abilities of Thai children. 

The objectives of this research were as 
follows: 

1. To develop a Thai version of the 
DOTCA-Ch encompassing its Orientation, 
Spatial Perception, and Thinking Operations 
sub tests using the WHO back-translation 
process; 

2. To examine the internal consistency, 
inter-rater reliability and test-retest reliability 
of the Thai DOTCA-Ch version of the Orien- 
tation, Spatial Perception, and Thinking Oper- 
ations sub tests. 


2. MATERIALS AND METHODS 


The DOTCA-Ch was designed to pro- 
vide a measurement of cognitive intervention 
need for children who are referred for treat- 
ment as a result of possible developmental, 
cognitive, or academic and learning difficul- 
ties, and for children with brain injuries as 
well as mild intellectually impaired children 
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(Katz et. al., 2004). For this reason, in this 
study the criteria of participants in purposive 
sampling were the students who were evalu- 
ated by medical and educational screening 
tests as mild learning problem students. Also, 
these participants did not have any physical, 
visual, or hearing disabilities. Participants in 
this study were 38 children who were 6-12 
years old, 18 male students (47.36%) and 20 
female students (52.64%), who were selected 
by purposive selection. The test group was 
made up of representatives from 17 learning 
disability students and 21 mild intellectual 
disability students. The average age of the 
participants was 8 years and 7 months. 

The methodology of this study consisted 
of three phases as is presented in Figure 1. The 
first phase was a process of translation and back 
translation of the assessment tool “DOTCA- 
Ch”, a dynamic criterion-referenced assess- 
ment of cognitive abilities and learning poten- 
tial for children with cognitive and learning 
difficulties. In the first phase, there were five 
steps which referred to WHO’s back transla- 
tion process (WHO, 2014) as follows: 

Step 1: The researcher asked for permis- 
sion from the Maddak Corporation in USA, 
who is owned of DOTCA-Ch, for the transla- 
tion assessment tool DOTCA-Ch. 

Step 2: The target of the language trans- 
lation procedure was to be able to translate 
the assessment tool from English to Thai. The 
characteristics of the translators were exper- 
tise in English and Thai, experience with 
the DOTCA-Ch assessment method, knowl- 
edge of occupational therapy, and expertise 
in developmental theory and performance of 
cognitive domains Orientation, Spatial Per- 
ception and Thinking Operation. 

Step 3: The Thai translation of DOTCA- 
Ch was back-translated into English by two 
experts from the Faculty of Western Language 
and the Faculty of Humanity of Chiang Mai 
University. This English version was then 
checked and proofed by the selected experts. 
The characteristics of the experts were the fact 
that they were fluent in English and Thai, had 
not worked in occupational therapy, and had 
not used the DOTCA-Ch assessment tool. 

Step 4: The content of Thai version of 
the DOTCA-Ch was cross-checked with the 
English version and corrected and hereby 
amended as needed. 

Step 5: The researcher trained two 
research assistants in Thai DOTCA-Ch Ori- 
entation, Spatial Perception and Thinking 
Operation sub tests before conducting a pre- 
liminary trial with three children to achieve 
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testing proficiency. 

The second phase of the study was pilot 
use of the method. This phase was the trial 
use of a Thai version of the DOTCA-Ch by 
the researcher and two research assistants to 
test the students with cognition and learning 
disability. During the trial use, the researcher 
and research assistants took notes about the 
problems of using this assessment tool. In 
the subsequent phase, the researcher started 
by contacting the Wat-Gukham School, San- 
patong District, Chiang Mai Province to get 
written permission for its use as a research 
area. After that, the researcher and research 
assistants administered the Thai DOTCA-Ch 
to the children with the cognitive disabil- 
ity who was studying in the Wat-Gukham 
School. The duration of the assessment was 
approximately 45 minutes. However, during 
the test sessions, if the children complained, 
were tired or had a lack of concentration, they 
were able to bring to an end to rest for about 
2 or 3 minutes. In addition, research assistants 
took notes and commented when they saw any 
mistakes or problems and wrote their sugges- 
tions after they had tried to use the DOTCA- 
Ch with seven students. Then, the researcher 
rectified and improved the content of the Thai 
DOTCA-Ch assessment tool according to the 
recorded comments and suggestions. 

Finally, the third phase was the process 
of checking on the accuracy of the assessment 
tool by determining the internal consistency, 
accuracy between examiners and stability in 
repeating the test. In this phase, the researcher 
started by contacting the Wat Chang Kian 
School and Kawila Anugul School to get writ- 
ten permission for their use as experimental 
areas. Then, the researcher used the modi- 
fied Thai DOTCA-Ch to appraise 38 children, 
17 learning disability students from the Wat 
Chang Kian School and 21 mild intellectual 
impairment students from the Kawila Anugul 
School. When the assessment was completed, 
the researcher determined the accuracy of 
inter-rater reliability between one of research 
assistants and the researcher who knew about 
the DOTCA-Ch and had previously used the 
Thai DOTCA-Ch version of the Orientation, 
Spatial Perception and Thinking Operation sub 
tests. Next, the researcher analyzed the data to 
assess internal consistency by the Conbach’s 
alpha coefficient and the intraclass correlation 
coefficient. Additionally, the researcher used 
the Thai DOTCA-Ch to appraise the same 
group of children to determine the test-retest 
reliability. The period between the test and the 
retest was 2 weeks. 
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Figure 1. The process in the methodology of this study 
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3: RESULTS AND DISCUSSION 


In this study, the results indicated that 
the Thai DOTCA-Ch version of the Orienta- 
tion, Spatial Perception and Thinking Opera- 
tion sub tests had an overall highly significant 
internal consistency coefficient of .82 (Table 
1). Each sub test also had high internal con- 
sistency coefficients of between .81 - .84. The 


overall inter-rater reliability was .80. In each 
sub test, the inter-rater reliability was between 
.72 - .84. This result implied that the assess- 
ment tool had a good to excellent level of 
reliability. The overall test-retest reliability 
was.85. In each sub-test, the inter-rater reli- 
ability was between .82 - .87, which meant 
that the Thai DOTCA-Ch version had excel- 
lent reliability. 


Table 1. The internal consistency of the Thai DOTCA-Ch version of the Orientation, Spatial Perception and Thinking 
Operation sub tests by the Cronbach’ alpha coefficient, Inter-rater Reliability and the Test-retest Reliability 


Sub test Cronbach’s alpha 
coefficient 

a. 

1. Orientation -83 
1.1 OP 82 
1.2 OT 84 
2. Spatial Perception 82 
2.1 SP1 81 
2.2 SP2 82 
2.3 SP3 83 
3. Thinking Operation -82 
3.1CA 82 
3.2RU 82 
3.3 RS 82 
3.4 PS1 81 
3.5 PS2 81 
3.6 GS1 82 
3.7 GS2 82 
All Subtests -82 


Inter-rater Test-retest 
Reliability Reliability 

cc cc 

83 .84 

84 85 

-82 83 

.84 86 

83 87 

82 86 

86 85 

.74 .85 

72 82 

715 86 

74 85 

75 .87 

-72 82 

715 86 

bP) 87 

-80 .85 


Note: OP = Orientation for Place, OT = Orientation for Time, SP1 = Directions on Child’s Body, SP2 = Spa- 
tial Relations between Children. and Objects in Near Space, SP3 = Spatial Relations on a Picture, CA = Categori- 
zation, RU = ROC Unstructured, RS = ROC Structured, PS1 = Pictorial Sequence A, PS2 = Pictorial Sequence B, 
GS1 = Geometrical Sequence A, GS2 = Geometrical Sequence B 
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All three sub-tests had intra-class corre- 
lation coefficients between .72 - .86 and had 
an overall inter-class correlation coefficient 
of .80. According to the criteria of Cicchetti 
and Sparrow (1981), the Thai DOTCA-Ch 
version of the Orientation, Spatial Perception 
and Thinking Operation sub tests had excel- 
lent reliability. For the Spatial Perception sub 
test, the inter-rater reliability had the highest 
intra-class correlation coefficient which was 
similar to the DOTCA-Ch original version 
(ICC = .95) (Katz et. al., 2004). The value of 
inter-rater reliability of the Spatial Perception 
sub test was higher than the Orientation and 
Thinking Operation sub tests. These results 
might have been the result of a number of fac- 
tors, such as that the two examiners might not 
have truly understood this assessment tool and 
had not gained adequate experience in using 
it in the assessment trials. Additionally, the 
methods of evaluation and scoring of the chil- 
dren came from them answering “yes” or “no” 
questions. This result may be related to the 
study of Ziviani et al. (2004) which revealed 
that a dichotomous scale assessment would 
have had a higher reliability than a multiple- 
scale assessment or an ordinal scale assess- 
ment. For example, if the child answered cor- 
rectly, he would get 1 point. On the other hand, 
if incorrect, he would get 0 points. This test- 
ing approach was unproblematic for the exam- 
iners in scoring so that it may have led to a 
lower reliability. The Thinking Operation sub 
test had a lower reliability than the other sub 
test . This was similar to the findings of (Katz 
et. al., 2004) who studied the original version 
of the DOTCA-Ch and showed that the inter- 
rater reliability was .87; this value indicated a 
lower reliability than Orientation and Spatial 
Perception sub tests. The reason for this result 
might be the different characteristics of the 
two examiners; one might have been stricter or 
more flexible in the observation of children’s 
behaviors than the other. Since the Thinking 
Operation sub test information is derived from 
observing the student’s performance and from 
interviewing the students, it is possible that the 
examiners could have evaluated student per- 
formance in the interview differently. These 
factors may have led to the lower reliability of 
the Thinking Operation sub test. However, all 
three sub tests had reliabilities ranging from 
good to excellent levels that were similar to 
the original version of the DOTCA-Ch. For 
this reason, the Thai DOTCA-Ch version of 
the Orientation, Spatial Perception and Think- 
ing Operation sub test could be considered to 
have a high inter-rater reliability (Cicchetti 
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and Sparrow, 1981). 

From the analysis of the test-retest 
reliability, the results showed that the intra- 
class correlation co-efficient was between .82 
- .87, and the overall intra-class correlation co- 
efficient was .87. This meant that the assess- 
ment tool had excellent reliability, according 
to the criteria of the reliability co-efficient 
of Cicchetti and Sparrow, 1981. In addition, 
there was an interval of 2 weeks between 
the test to the retest evaluation so that most 
likely, this duration did not have an impact 
on the cognitive development of the children. 
According to the theory of Piaget, cognitive 
development can be categorized into four 
levels, including a sensorimotor stage, preop- 
erational stage, concrete operational stage and 
the formal operational stage. The basic levels 
will develop the higher levels, and this process 
takes more than 2 years to complete. As a con- 
sequence, the period of 2 weeks did not cause 
any discernable differences in development. 
The lower test-retest reliability of the orienta- 
tion sub test might have been caused by the 
characteristics of the questions for which the 
children needed to provide answers by expla- 
nation their reasons. For instance, in the ques- 
tion of place orientation, “Where do you live? 
Explain in detail about where do you live.”, 
the child might answer differently each time 
it is requested. The scoring of these questions 
had three levels: 0 point, 1 point, and 2 points. 
Zero points meant the child did not know the 
answer, 1 point meant the child answer cor- 
rectly by choosing from multiple choices, or 
the examiner explained the question in more 
detail, and 2 points meant the child answered 
correctly by themselves without being given 
choices, or without any explanation from the 
examiner. As a result, the answers from the 
retest might have been from memory of the 
first answer. In fact, the child might not truly 
have got the question and answered with the 
examiner’s explanations or choices which 
gave | point. However, when they were 
given a retest, they might have gotten 2 points 
because they remembered the choices or the 
explanations from the first test. Thus, although 
they performed the identical, in each type of 
question they got different points for each. 
This may have affected the test-retest reliabil- 
ity of the Orientation sub test slightly less than 
the Spatial Perception and Thinking Operation 
sub test. However, all these sub tests had high 
test-retest reliabilities. These results were sim- 
ilar to those obtained by Ziviani et al. (2004) , 
who found that the test-retest reliability of the 
original version of the DOTCA-Ch was high 
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to very high. On the other hand Daibes (2012) 
found that regardless of the fact that 53% of 
the assessment items of the original version 
of the DOTCA-Ch had a high test-retest reli- 
ability, 19% of the assessment items had a low 
test-retest reliability. 

All of the above results indicated that 
the Thai DOTCA-Ch versions of Orienta- 
tion, Spatial Perception and Thinking Opera- 
tion sub tests had the psychometric proper- 
ties of consistency to be good assessment 
tools. Indeed, although the assessments were 
repeated, the results were no different. These 
results were not only focused on the end prod- 
uct of the That DOTCA-Ch versions but also 
gave guidelines for researchers in other coun- 
tries in which English is a second language to 
develop the standardized test in their own lan- 
guage by the WHO back translation process. 

A limitation of this research was that 
all tests of reliability came from the score 
points before using mediations. In future, 
the researcher may study the norms in Thai 
children and other tests of the validity of this 
assessment tool, such as, predictive valid- 
ity, construct validity and concurrent valid- 
ity. Moreover, interestingly, future research 
can study the psychometric properties of the 
DOTCA-Ch after using mediation. 


4. CONCLUSIONS 


This research showed the process of 
cognitive assessment tool development in the 
context of a country where English is a second 
language. This systematic process developed 
a new assessment methodology which had 
excellent reliability. Suitable cognitive assess- 
ment was enabled that overcame any language 
barrier so that the cognitive problems of chil- 
dren could be accurately identified. 


ACKNOWLEDGEMENTS 


This project was funded by The Grad- 
uate School, Chiang Mai University, Thailand. 


Conflict of interests 


Authors declare no conflict of interest. 


REFERENCES 


Chinchai, S. (2000). Basic principle of occupational 
therapy. Chiang Mai: Occupational therapy 
department, Chiang Mai University. 

Cicchetti, D. V., & Sparrow, S. S. (1981). Developing 
criteria for establishing interrater reliability of 
specific items: Applications to assessment of 
adaptive behavior. American Journal of Mental 
Deficiency, 86, 127-137. 

Donaghy, S., & Wass, P. J. (1998). Inter-rater reliabil- 
ity of functional assessment in a brain injury 
programme. Archive of Physical Medicine and 
Rehabilitation, 9, 1231-1236. 

Daibes J.S.A. (2012). The Dynamic Occupational 
Therapy Cognitive Assessment for children 
DOTCA-Ch: Pilot study of inter-rater and Test- 
retest reliability (Dissertion). Palestine: Nables, 
AnNajah National University. 

Encyclopedia of Children’s Health (2014). Cognitive 
development. On-line. Available from Internet, 
http://www.healthofchildren.com/C/Cognitive- 
Development.html, Accessed 16 May 2014. 

Hinkle, D. E., Wiersma, W., & Jurs, S. G. (2003). 
Applied Statistic for the Behavioral Sciences. 
5th ed. Boston: Boston: Houghton Mifflin. 

Josman, N. (2005). The dynamic interactional model in 
schizophrenia. Cognition and occupation across 
the life span: Models for Intervention in Occu- 
pational Therapy. 2nd ed. Bethesda: American 
Occupational Therapy Association. 

Katz, N. (2005). Cognition & Occupation across the 
Life Span. 2nd ed. Bethesda: The American 
occupational Therapy. : Association Inc. 

Katz, N., Golstand, S., Traub Bar-Ilan R, , & Parush,S. 
(2007). The dynamic Occupational Therapy 
Cognitive Assessment for Children (DOTCA- 
Ch): A new instrument for assessing learning 
potential. American Journal of Occupational 
Therapy, 61, 41-52. 

Katz, N., Kizony, R. & Parush, S. (2002). Visuomotor 
organisation and thinking operations perfor- 
mance of school-age Ethiopian, Bedouin, and 
mainstream Israeli children. The Occupational 
Therapy Journal of Research, 22,34—43. 

Katz, N., Parush, S., & Traub Bar-Ilan R, (2004). 
DOTCA-Ch - Dynamic Occupational Therapy 
Cognitive Assessment for children manual. 
Pequannock NJ.: Maddak Inc. 

Lefrancois, G. R. (2006). Theories of Human Learning. 
Belmont, CA: Thomson Higher Education. 

Munkhetwit, P. (2008). Occupational therapy interven- 
tions for persons with cognitive-perceptual dys- 
function. Chiang Mai.: Darawan Publishing. 

Nowack, R. A. (2005). Performance Comparison of 
Boy Versus Girls between the Ages of 9-12 on 
the DOTCA-Ch: A pilot Study. NY: Thesis Touro 
College School of Health Sciences Occupational 





www.ijcrsee.com 


74 


(IJCRSEE) International Journal of Cognitive Research in Science, Engineering and Education 
Vol. 2, No.1, 2014. 


Therapy Program. 

Rodger S., Daley E., Hughes K. & Ziviani J. (2005). 
Dynamic Occupational Therapy Cognitive 
Assessment for Children: Perceived utility in 
Australian occupational therapy practice. Aus- 
tralian Occupational Therapy Journal, 52, 
337- 34. 

Shettleworth, S. J. (2010). Cognition, Evolution and 
Behavior. 2nd edition. New York: Oxford Uni- 
versity Press. 

Wittayakorn, S. (2004). Occupational therapy in chil- 
dren: clinical evaluation. Chiang Mai: Occu- 
pational therapy department, Chiang Mai 
University. 

World Health Organization. (2014). Process of trans- 
lation and adaptation of instruments. Online. 
Available from Internet, http://www.who.int/ 
substance_abuse/research_tools/translation/en/ 
index.html, Accessed 15 January 2014. 

Ziviani, J., Rodger, S., Pacheco, P., Rotsey, L., Smith, 
A., & Katz, N. (2004). The Dynamic Occupa- 
tional Therapy Cognitive Assessment for Chil- 
dren (DOTCA Ch): Pilot Study of Inter-rater and 
Test Retest Reliability. New Zeland Journal of 
Occupational Therapy, 51(2). 


www.ijcrsee.com 
a 


(IJCRSEE) International Journal of Cognitive Research in Science, Engineering and Education 
Vol. 2, No.1, 2014. 


www.ijcrsee.com 


76 


